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(54) Resolution enhancement by synthesis of scan signals 

(57) Digital image data of high resolution are 
obtained by synthesis of the scanning signals of two 
scanners, which scan an original document with mutu- 
ally different lower resolutions. 

The synthesis comprises transforming the scan sig- 
nals of the first scanner into a first spectrum and trans- 
forming the scan signals of the second scanner into a 
second spectrum, superposing in predetermined man- 
ner versions of the first and second spectrum shifted 
over the spectral axis, and deriving a third spectrum 
from the result thereof. Retranstormation of the third 
spectrum finally gives the required digital image data of 
high resolution. 
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Description 



[0001 ) The invention relates to a method and apparatus for generating digital image data, relating to an image, as is 
the case, for example, in a digital copying machine or in a free-standing scanner apparatus. 

s [0002] In apparatus of this Kind, a document having an image thereon is scanned by means of an electro-optical con- 
verter to generate digital image data which contain a grey value tor each image element or pixel. This is usually dona 
with a CCD array, on which a part of the document in the form of a line is projected by an optical imaging system. By 
moving either the document or the imaging system in a direction perpendicular to the said line part of the document, 
the document is scanned completely, line-by-line, in accordance with a usually rectangular raster of pixels. 

10 [0003] The spatial density or resolution of the pixels is in practice of the order of some hundreds of pixels per inch, 
usually specified as "dpi" (= dots per inch). 

[0004] For a good description of an Image it is of course desirable that pixels ^ould have a high spatial density. How- 
ever, high-resolution systems are expensive, both because of the requirements relating to the CCD array and the 
requirements relating to the optical imaging system. There is therefore a demand for a scanner system which generates 
rs image data at high resolution and is yet constructed from simple and hence inexpensive components. The present 
invention meets this demand 

(0005] To this end, the method according to the invention comprises the following steps: 

1] generating first digital image dam by scanning the said image with a scanner having a first resolution f1 ; 
90 2) generating second digital Image data by scanning the sajd image with a second scanner having a second reso- 
lution t2 higher than ft ; 

3) and combining the said first and second digital image data to form third digital ima.ge data, which describe the 
sad image with a third resolution f3 wrwch is higher than f i and higher than te. 

ss [0006] According to one embodiment, step 3) comprises the following sub-steps: 

a) transforming the said first digital image data f nto a first spectrum, and transforming the said second digital image 
data Into a second spectrum, which spectra have an amplitude on a spectral axis; 

b) superposing in a predetermined manner versions of the first and second spectrum shifted over the spectral axis. 
so to grye a first combination; 

c) deriving a third spectrum from the first combination; 

d) and re-transforming the third spectrum, the result being interpreted as the third digital image data. 

[0007] The invention is based on combining the image data of two scanners, each of which has a relatively low res- 
55 oiution different from the other, to give image data of a higher resolution. This combination is possible in the frequency 
domain. The digital image data are therefore first transformed thereto with a Fourier transform, whereafter they are . 
processed further. Alternatively, it is possible to subject the image data to a cosine transform and carry out the combi- 
nation in the cosine domain. 

[0008] The said resolutions are preferably so selected that 

40 

f3«fi +fz-G(fl,f2) 

where G(fiLb) is the highest common divisor of a and b. On the basis of theoretical considerations, this value of t3 is the 
highest frequency at which the results of the method still form a reliable estimate of the original image, i.e., still corre- 
43 spond to the original image, H the latter were scanned at the resolution f3. There is therefore no point in mating f3 
higher, bacausB it does not yield any more information On the other hand, it is also undesirable to make f3 lower, 
because then the information present in the image data Is not used to maximum effect 

[0009) As a marginal condition for the values of f 1 and f2, their ratio must be a rational ratio because it is only in that 
case that the theory on which the processing of the digital image data from the scanner ts based is valid. However, this 
so does not form any limitation, because this is always the case as a result of the discrete form of the scanners, 

[001 0] It should be noted that operations in the frequency domain on image data of two scanners each having a res- 
olution different from the Other is Known per se. These operations relate mostly to filtering image data in order to remove 
disturbing elements from the digitised image. 

[001 1 ] In this connection, US-A-5 1 21 445 describes a method aimed at removing moire patterns. Patterns of this kind 
55 occur when a rastered image is scanned with a scanner, due to the fact that the raster frequency of the image interferes 
with the scanning frequency (resolution) of the scanner. The interference causes sidebands on either side of the spec- 
trum of the original non-rastered image, in the frequency domain. Also, the entire spectrum, including the sidebands, is 
periodically repeated as a result of the discrete scanning by the scanner. The said sidebands contain information on 
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bolh the restoring and the scanning. The known method comprises eliminating the side bands as far as possible by 
comparing the spectra of the image data from the two scanners, so that the original non-rastered image can be recon- 
structed. Thus this method not only removes the moire but also the raster. Effectively, therefore, information present in 
the scanned image is destroyed. The precise object of the present invention is to retain as far as possible aD the infor- 
5 matron in the scanned image. 

[001 2j The invention wf U now be explained with reference to the tallowing Frgures, in which like references refer to like 
elements. The exemplified embodiment in this explanation is, however, only one example of one possble implementa- 
tion. It will be clear to the skilled man that other embedments of the invention are possible within the scope of the 
claims. 

ro [001 3] In fte drawings: 

Fig. 1 is a diagram showing the principle of the scanner apparatus according to the invention. * 
Fig, 2 is a block schematic of the apparatus according to the invention. . 
Fig. 3 A is a simplified spectrum of an original image. 
Fig, 3B is a spectrum of a scan signal of an image. 

Fig. 4 is a diagrammatic representation of the process of reconstructing a high-resolution signal from the two lower- 
resoiution scan signals in accordance with the invention. 
h Figs. 5A - F are a visualisation of the operations on the spectra during the reconstruction. ' 

so [001 4] Fig. 1 is a sketch showing the principle of a scanner according to the invention, An original document 1 is trans- 
ported in a direction 2 at a unffbrm speed. A (ens 3 images the document on two CCD arrays 4 and S, which scan a line 
area, perpendicular to the direction of transport 2, of the document signals being generated in accordance with the 
grey values of image parts or pixels. These signals are converted to digital image data by means of A/D converters. 
Since the document is transported along the imaging system, the entire document is scanned line-by-line, 

ss [0015] Generally; a combination of a CCD array and a lens is also termed a "camera". The scanner device of Fig. 1 
thus comprises two cameras. 

[001 6 J Array 4 scans the document with a resolution f 1 and array 5 does so with a resolution f2. which is different from 
f 1 . This can be effected by using CCD arrays having different CCD density, and also by making the imaging system (the 
lens) different for the arrays, so that one array scans the document with a different enlargement from the other. 

30 [001 71 Instead of the lens in Fig. i , it is also possible to use a seffoc lens array (an array of imaging optical fibres), 
[001 8] It is also possible to scan the document twice with different magnification, using one camera having variable 
magnification, and to store the signals of the two scans in a memory. The signals with different resolution are then not 
generated simultaneously, but can be delivered in synchronism final ry by reading out of the memory simultaneously or 
seml-slmuftaneously the signals corresponding to the same line position. 

35 [0019] Fig. 2 is a block schematic of the apparatus according to the invention in the embodiment of Fig. 1 . The CCD 
arrays 4 and 5 are each connected to an A/D converter 11.12 respectively, to convert their analogue signals into digital 
image data. The A/D converter 1 1 is connected to the delay device 13 tor synchronising the image data, so that the 
image data delivered in the two channels relate to the same linear area on the scanned document. The delay device 1 3 
is connected to a DFT module 14 and the A/D converter 12 is connected to a DFT module 15. A DFT module performs 

40 a discrete Fourier transformation on the image data. The DFT modules are each connected to a synthesis module 16. 
This is intended to combine the transformed image data from the two channels into transformed image data having a 
higher resolution f3 in the frequency domain. The operation of the synthesis module will be desc/bed hereinafter. The 
synthesis module 16 is in turn connected to an inverse DFT module 17 tor transforming the data from the synthesis 
module into digital image data having the higher resolution f3. 

4S [0020] The apparatus described here can be constructed as an independent scanner device for scanning documents 
and in so doing generating digital image data, which are then delivered via an electrical connection to, for example, a 
workstation or computer. The apparatus can also form part of a digital copying apparatus, in which the scanner signals 
are converted to control signals for a printing apparatus, whereby the latter prints a copy of the document on an image 
support such as a sheet of paper. The structural elements required tor the construction as shown in Fig. 1 , although 

so not described above, are generally known to the skilled man 

[0021 ] Before discussing the procedure in processing two image signals of spatial frequencies f 1 and f2 respectively, 
to form an image signal of spatial frequency f3, reference will first be made to Figs, 3A and B. which show the effect of 
scanning on the image signal. 

[00(22] The basis taken in the following will be an original image, the spatial progress of the image information of which 
ss will be designated the "original signal h\ The spectrum of "h" will be designated "H M . 

[0023] Fig. 3A shows a (simplif ied) spectrum (H) 21 of the original image. When this image is scanned whh a spatial 
frequency ta, a scan signal is generated with a spectrum as shown in Fig. 3B. This spectrum contains the original spec- 
trum 21 and periodic repetitions 22 of this spectrum on the multiples of the scanning frequency fa. When the scanning 
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frequency fa is sufficiently high compared with the extent of the original spectrum, the original image signal can be 
recovered faultlessly from the spectrum of the wanning signal If the bandwidth of h is greater than fa/2, then the peri- 
odic repetitions of the original spectrum will overlap one another and distortion will occur so that the original image sig- 
nal h can no longer be recovered faultlessly. This Is known as the Nyquist theorem in signal theory 
s [0024] The spectrum H A oi an original signal h scanned at a frequency fa can now be described as: 

go 

H«« £ T k '*H 

10 

wherein His the spectrum of hand 

7* is a shift operator defined as: (T a f)(tH(trs) . 

where f is any function. 

T5 [0025] This formula is the mathematical representation of what is shown in Fig. 3B. 

[0026] The process of reconstruction of an approximation of the original signal h from the two scanning signals or. 
more generally, of the processing of two image signals of frequencies f 1 and f2 respectively, to form an image signal of 
frequency f3, is shown irrFig. 4. tt progresses in lour steps! In the first step, the spectra of the scanning signals are cal- 
culated by means of a Fourier transform. In the second step.a superposition of the spectra of the scanning signals is 

20 calculated. In the third step, the result of the superposition te unravelled so that (by approximation) the spectrum H of 
the original signal remains. The (approximated) original signal is derived from this in the fourth step by inverse Fourier 
transformation, so that it would appear as if scanning was carried out at the increased frequency f3. 
[0027] This procedure will now be explained by reference to a numerical example. The general case will be discussed 
hereinafter. * 

S5 [0028] Two scanning frequencies f 1 = 200 dpi and f2 - 300 dpi are used for this example. 

[0029] The various steps of the process as described with reference to Fig. 4 w3l now be described in greater detail. 

First step 

*so [0030] Fig. 5A shows the spectrum H of an original image h obtained by subjecting the latter to a Fourier transform. 
Figs. 5B and 5C respectively show the spectra H 1 and hfe respectively of the two scanning signals associated with the 
scanning frequencies f 1 and f 2 respectively, tt will be clear that these spectra are not suitable for reconstructing the orig- 
inal signal h faultlessly therefrom, due to the overlapping of the periodic repetitions. 

35 Second step 

For the purpose of explanation, an auxiliary frequency fx will first be introduced, which is defined as: 
fx « K(f 1 , f2) , where K denotes the lowest common multiple. 
[0031] In the example, therefore, fx ■ 600 dpi. 
4o [0032] The following superposition s is selected for the example described; 



1 - H * 



10033] This choice Is based on the following considerations. Let us assume that the original signal h was scanned at 
4$ a frequency fx. i.e. 600 dpi, then the spectrum of the scanning signal was: 

h * = £ t600Kh 

k 

SO 

[0034] It will readily be seen that 

H 1 o(HT 200 + T 400 )H )( 

SB 

H^l+T 300 )^ 

[0035] In order to avoid overlapping as much as poesfele, there is selected for the superposition a combination in 
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which the shift differences are kept as small as posstole. A good choice therefore is the above-mentioned combination, 
because in that case: 

T- 100 H 1 -H 2 -fT l00 ^+T 100 )H )( 

[0036] One period t& now cut out of the result 8 of the superposition by making equal to zero all the spectra values 
outside (-fx/2, fx/2), in this example: (-300. 300). The result of this is shown in Fig, 5D. 

[0037] When the bandwidth of the original signal h is less than 200 dpi, generally (fl + fZ - G(f1. f2)V2 . where G 
denotes the largest common divisor, this equation also applies when H„ is replaced by H. because then there is no over- 
io lapping of spectra. Jn the case of larger bandwidths the resulting signal is distorted. 

The resulting spectrum is then unravelled by again subjecting it to a combined shift operator. A good choice for this 
15 operator appears to be: 

-200 — -100 -.100 — 200. 

(T + T ■ T 'T ) 

The result of this is shown in Fig. 5E. 
so [0038] A cut-out Is made from this spectrum between (-fx, 0), in this example (-600, 0}, whereafter a last translation 
T*** 2 , in this example T 300 * delivers an approximated spectrum H' of the original signal h. This Es shown In Fig. 5F. 

Fourth step 

2s An estimate of the original signal h can then be calculated from the spectrum H' using an inverse Fourier transform. To 
have the signal available at the required frequency f3 after transformation, the spectrum H* is first periodically repeated 
with period f3 by using the operator 



30 



so 



6$ 



2™ 

K— <Z3 

and then subjected to inverse Fourier transformation. 

In practice, this step is carried out In one operation by using a discrete inverse Fourier transform with the correct period. 
[0039] The process has been explained as an example hereinbefore for the case of fl = 200 dpi. 12 = 300 dpi. 
The process will now be described in general form hereinafter. 

The scanning frequencies should be so selected that f 1/12 is a rational ratio (this is always the case in practice due to 
the construction of the scanner with discrete CCD arrays). There are then natural numbers a and b. for which: 

M -G(f1,fc).a 

f2*G(f1,f2) .b 
fx»b.M «a.f2 
where G(f 1 . f2) is the largest common divisor of f 1 and 1Z. 
First step 

The first step comprises Fourier transformation of the signals obtained by scanning with scanning frequency fl and f2 
respectively, and is of course the same as the first step described above in the exemplified process. 

We then define an operator S ■ T^ M - ffi > 
and operators P a and P&, for which: 
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p fc * (i+s*+s 2 V..+s (B " 1)11 ) 

P h = (US b +S Zb +-.^ (,| - 1)b ) . 

5 then: 

H 2 = P b H 3 

10 

There are now operators A and B, for which: 

A.P. + B.P^GfP,^) 

75 G(P a . p b) is ft™ largest common divisor of the operators P m and P^ contained as polynomials in S. 

[Q040] A, B and G(P ai Pb) can be found by using fte Euclidean algorithm known in mathematics. In the above exem- 
plified case, this would give: 

G( p m p b) a1 -6 1+ ? 5 -S 7 +S 8 

so 

' A^ 1 
B = 1+S 3 

as The following is now taken as superposition s (again for the general case): 

e = S c *- 1)pa ^ jy2 <A.H 

One period Is now cut out of the result s of the superposition by making all the spectrum values outside (-fx/2, lx/2) 
- 30 equal to zero. 

If the bandwidth of the original signal h is limited within the value (f l+f2-G<fl ,f2))/2 , then the following also applies: 

^ s ^><^>*G(P, ( P b ).H 

as Third step 

The resulting spectrum is then unravelled by subjecting it to the following operator: 

40 

[0041 J From this spectrum a cut-out Is made between (-fx.Q), whereafter a last translation T 1 ^ gives an approximated 
spectrum H' of the original signal h, 

[0042] If the bandwidth of the original signal h is limited within lha value <f 1 +f2-G(f 1 »f2))/2 . then the unravelling yields: 
45 H'=(S"^ -S**)H 

so that H and h are exactly reconstructed. 
Fourth step 

so 

This is again the same as the fourth step described above. This completes the reconstruction. 

[00431 By means of the above description it Is possble. using two cameras having relatively low resolutions f 1 and f2 
which differ from one another, to construct a scanner device which can deliver digital image signals having a relatively 
high frequency f3. The latter imago signals are then synthesised from the signals ol the two cameras using the above- 
55 described method. 

[0044] It should be noted that the synthesized signal has a frequency content defined by f3 - (fl+f2-G(f l,f2)) . Select- 
ing a varus for f3 high Br than (f l+t2-G(fl,f2) ) does not yieW a sharper image than that for the value mentioned here. 
[0045] In the exampiB use was made of (discrete) Fourier transform. However, use can also be made of other orthog- 
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onal transformations, such as, for example, (discrete) cosine transform. The method is not changed as a result, 
although the operators used may differ, tt is within the scope of the sWfled man to adapt the operators for other trans- 
formations, tna skilled man in this case having a wide knowledge of both signal theory and mathematics. 
[0046] The scan signals can also be generated with two-dimensional CCD arrays of different resolutions, or one or 
5 two identical two-dimensional arrays with different magrulication optics. The image signals can be synthasised. similarly 
to the above-described method, by a suitable adaptation of the data processing, either by splitting the two-dimensional 
algorithm into one-dimensional operations, or by adapting the theory (or the two-dimensional case. 

Claims 

10 

1 . A method of generating digital image data relating to an image, comprising 

generating first digital image data by scanning the said image with a first scanner having a first resolution f 1 : 

75 generating second digital image data by scanning the said image with a second scanner having a second res- 

olution f2 higher than tl ; 

and combining the said first and second digital image data to form third digital image data, which describe the 
said image with a third resolution f3 which is higher than ft and higher than f2. 

2. A method according to claim 1 , 

wherein the combination of the said first and second digital image data comprises: 

a) transforming the said first digital image data into a first spectrum, and transforming the said second digital 
25 image data into a second spectrum, which spectra have an amplitude on a spectral axis; 

b) superposing In a predetermined manner versions of the first and second spectrum shifted over the spectral 
axis, to give a first combination; 

so c) deriving a third spectrum, from ttefirt corrttnafion: 

d) and re-transforming the third spectrum, the result being interpreted as the third digital image data. 

3- A method according to claim 2. 
as wherein an orthogonal transform is used in step a). 

4. A method according to daim 3, 

wherein the transform is a Fourier transformation. 

40 5. A method according to claim 3. 

wherein the transform is a cosine transformation. 

6. A method according to daim 2, 
wherein step c) comprises: 

45 

d) removing repetitions from the said first combination in such manner that one single period remains; 

C2) superposing copies of the said one single period, which copies are shifted over the spectral axis, to form a 
second combination; 

so 

c3) making a cut-out from the said second combination, in such manner that one single spectrum remains; and 

c4) shifting the tatter one single spectrum over ffie spectral axis over a distance determined in accordance with 
a predetermined criterion, in such manner that it comes to rest symmetrically on the spectral axis. 

as 

7. A method according to claim 6. 

wherein step d) is carried out by making the amplitude of the combined spectrum equal to zero for frequency val- 
ues less than -fx/2 and larger than fcc/2, wherein he is the lowest common multiple of f 1 and f2. 
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8. A method according to claim 6, 

wherein step c3) is carried out by making the amplitude of the said second combination equal to zero for frequency 
values less than -fx and greater than zero, wherein fx Is the lowest common multiple or 11 and 12. 

9 9. A method according to claim 7, 

wherein step c4) is carried out by shifting said one single spectrum over a distance fx/2, 
wherein fx is the lowest common multiple of f 1 and 12. 

10. A method according to claim 1. 

10 wherein the said resd uttons are so eel ected that 

!3=f1 +f2-G(M,f2) 
wherein G(a.b) fe the highest common divisor of a and b, 

is 

1 1 . A method according to claim 1 , 

wherein by approximation 11 ■ 200 dpi, f e 300 dpi and f 3 = 400 dpi. 

12. A method according to daim 1, 

so wherein by approximation f 1 = 300 dpi. f - 400 dpi and f 3 =■ 600 dpi. 

• 1 3. Apparatus for generating diojtai image data relating to an image, comprising 

means tor scanning the said image, with a first resolution f 1 , and in so doing generating first digital image data; 

25 

means for scanning the said image, with a second resolution t2, and in so doing generating second digital 
image data, which second resolution 12 is higher than M; 

and means for combining the said first and second digital image data, relating to a same portion of the image. 
so to give third digital image data, which describe the said image with a third resolution 13 higher than fl and 

higher than f2. 

14. Apparatus according to daim 13, 

wherein the means for scanning the image comprise a scanner constructed from an array of electro-optical can- 
as verters and means for imaging at least a part of the Image on the said array. 

15. Apparatus according to daim 14, 

comprising two said scanners, one of which scans the image with the said resolution 11 and the other scans the 
image with the said resolution 12. 

40 

1 6. Apparatus according to daim 14, 

comprising a scanner provided with means for scanning an image with different resolutions. 

in which apparatus the means for scanning the image utilise said scanner to generate the first and the second 
45 digital image data, 

and which apparatus is also provided with a memory for intermediate storage of digital image data, connected 
to the means for scanning and to the means tor combining the said first and second digital image data. 

so 17. Apparatus according to daim 13, 

wherein the means for combining the said first and second digital image data comprise: 

a) means tor transforming said first digital image data into a first spectrum, and transforming said second digital 
image data into a second spectrum. 

55 which spectra have an amplitude on a spectral axis. 

b) means for superposing in a predetermined manner versions of the first and second spectrum, shifted over 
the spectral axis, to gjve a first combination, 
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c) means tar deriving a third spectrum from the first combination, 

d) means tor retranstonrtng the third spectrum. 



$ 18. Apparatus according to daim 13. 

wherein by approximation 11 = 200 dpi, f = 300 dpi and f3 = 400 dpt. 

id. Apparatus accorcfing to claim 13, 

wheran by approximation ft « 300 dpi, f = 400 dpi and f3 - 600 dpi. 
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